Performance comparison of masking models based on a new psychovisual test method with natural scenery stimuli

Various image processing applications exploit a model of the human visual system (HVS). One element of HVSmodels describes the masking-effect, which is typically parameterized by psycho-visual experiments that employ superimposed sinusoidal stimuli. Those stimuli are oversimplified with respect to real images and can capture only very elementary maskingeffects. To overcome these limitations a new psychovisual test method is proposed. It is based on natural scenery stimuli and operates in the wavelet domain. The collected psycho-visual data is finally used to evaluate the performance of various masking models under conditions as found in real image processing applications like compression. Keywords— HVS; Masking; Wavelet; Image Compression; Quality Assessment I. History of Masking Models and Measurements VARIOUS image processing applications exploit the limitations and imperfections of human visual perception. Typically, a model of the human visual system (HVS) predicts the visibility of specific image stimuli, so that for example a watermarking scheme can more efficiently hide its information in the image, a compression scheme achieve a better visual quality or a printer deliver better rendered prints due to HVS-improved halftoning patterns. However, the main difficulty lies in a proper description of the human visual system. It is of amazingly high complexity. Hence, the idea to model the entire system by a direct transcription from physiological features to a numerical model was very soon abandoned. It is rather tried to parameterize psycho-physical effects, like limited spatial frequency resolution, light adaptation or masking. Generally speaking, masking refers to the reduced or totally inhibited visibility of a signal stimulus due to the simultaneous presence of a masker stimulus. However, there exist several types of masking [11]. Contrast masking occurs if a weak signal is hidden by a stronger signal that is present at the same location. One speaks of texture or activity masking, if a signal is hidden by the local image surround that is so busy and irregular that the HVS gets confused and is no longer able to localize the initial signal. Both masking phenomena are of great relevance for image compression or image quality assessment tools. Before these masking effects can be exploited in final image processing applications, they need to be measured and modeled. The “classical” masking measurements employ two superimposed sinusoidal gratings. Their contrast, spatial frequency and orientation is varied to determine the conditions when masking occurs. Based on this data a model is built that predicts the masking effect. Typically, it processes the image by several filter-channels that differ in their band-pass frequency and spatial orientation. The model might only consider the masking between stimuli of the same spatial frequency and orientation [12], called intra-channel-masking, or also between different orientation and frequencies [9, 18, 19], called inter-channel masking. The variety of channels that is considered to compute the inhibitory term can also account for the interaction between the luminance and chrominance channels [4, 5, 13, 17]. The subjective decision task whether masking occurs or not is typically modeled as a Gaussian process [10] to explain the varying threshold level of detection. There are two drawbacks of the “classical” method that is based on sinusoidal patterns: First, the sinusoidal masker grating represents a tremendous simplification with respect to natural scenery images. Thus, effects like texture masking cannot be measured. Even if more recent measurements employ noise patterns, they still represent an oversimplification with respect to natural images. Second, in general, a sinusoidal stimulus is much simpler than for example a real compression artifact. Hence, the finally derived model might do a good prediction for sinusoidal patterns, but not for real images and thus is of limited usage for image processing applications. A first step towards masking measurements with realistic distortions and natural scenery is presented in [3, 7], where nondeterministic noise-patterns were used. Also Watson [21] did some preliminary experiments with more complex backgrounds. However, the masking measurements are still not as close to real conditions as they could be. Therefore, the authors propose a new scheme of masking measurements. Natural scenery images are decomposed by a wavelet-decomposition and quantized within one subband of this decomposition to generate the test stimuli. This way the masker-signal is a real image and the test signal a quantization error signal, like it would occur in image compression applications. This new technique allows to measure the contrast masking and texture masking effect. Finally, a database with the results of the masking experiments is provided. The Modelfest group [8] established a similar database, but for sinusoidal test stimuli. Once a common database is available, it can be used for the performance evaluation of various masking models. Such a comparative work is of urgent need, because typically only new measurements and models are presented without a clear performance comparison to existing ones. However, the only comparative work known to the author is based on sinusoidal test patterns [20] and thus of limited relevance for real image processing applications. This article presents a comparison of five commonly used masking models based on the psychovisual measurements with natural stimuli. The paper is structured as follows: Section II describes how the new test stimuli are generated. In Section III the psychovisual testing procedure is explained. Section IV explains the process of the masking model training/optimization using the data from the psycho-visual experiments. Section V defines the masking models that are compared in their performance. Finally, in Section VI the performance of these models is analyzed and discussed, before the final conclusions are drawn.